{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Azure AI Search with Cohere Embed V3 Int8 Support\n",
    "This code demonstrates the use of the Cohere API to generate embeddings with the latest and highest-performing model from Cohere, [Cohere Embed V3](https://txt.cohere.com/introducing-embed-v3/). It also explains how to store these embeddings in Azure AI Search using the Python SDK as an Int8 vector data type. This approach maintains 100% of the embedding quality while reducing our index size by 4x compared to the Float32 representation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set up a Python virtual environment in Visual Studio Code\n",
    "\n",
    "1. Open the Command Palette (Ctrl+Shift+P).\n",
    "1. Search for **Python: Create Environment**.\n",
    "1. Select **Venv**.\n",
    "1. Select a Python interpreter. Choose 3.10 or later.\n",
    "\n",
    "It can take a minute to set up. If you run into problems, see [Python environments in VS Code](https://code.visualstudio.com/docs/python/environments)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install -r azure-search-cohere-embed-v3-sample-requirements.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cohere\n",
    "import numpy as np\n",
    "import os\n",
    "from dotenv import load_dotenv\n",
    "from azure.search.documents.indexes import SearchIndexClient\n",
    "from azure.search.documents import SearchClient\n",
    "from azure.search.documents.models import (\n",
    "    VectorizedQuery,\n",
    ")\n",
    "from azure.search.documents.indexes.models import (\n",
    "    HnswAlgorithmConfiguration,\n",
    "    SearchField,\n",
    "    SearchableField,\n",
    "    SearchFieldDataType,\n",
    "    SearchIndex,\n",
    "    SimpleField,\n",
    "    VectorSearch,\n",
    "    VectorSearchAlgorithmKind,\n",
    "    VectorSearchProfile,\n",
    ")\n",
    "from azure.core.credentials import AzureKeyCredential\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set Up Cohere and Azure Credentials\n",
    "Before generating embeddings or interacting with Azure AI Search, we need to set up our credentials for both Cohere and Azure AI Search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv()\n",
    "cohere_api_key = os.getenv(\"COHERE_API_KEY\")\n",
    "co = cohere.Client(cohere_api_key)\n",
    "\n",
    "search_service_endpoint = os.getenv(\"AZURE_SEARCH_SERVICE_ENDPOINT\")\n",
    "search_service_api_key = os.getenv(\"AZURE_SEARCH_ADMIN_KEY\")\n",
    "index_name = \"cohere-embed-v3-index\"\n",
    "credential = AzureKeyCredential(search_service_api_key)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate Embeddings Function\n",
    "This function will use the Cohere API to generate int8 embeddings for a list of documents. These embeddings are optimized for search document use cases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_embeddings(texts, input_type=\"search_document\"):\n",
    "    model = \"embed-english-v3.0\"\n",
    "    # Ensure texts is a list\n",
    "    if isinstance(texts, str):\n",
    "        texts = [texts]\n",
    "\n",
    "    response = co.embed(\n",
    "        texts=texts,\n",
    "        model=model,\n",
    "        input_type=input_type,\n",
    "        embedding_types=[\"int8\"],\n",
    "    )\n",
    "    return [embedding for embedding in response.embeddings.int8]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create or Update Azure AI Search Index\n",
    "This function creates or updates an Azure AI Search index to include a vector field for storing the document embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_or_update_index(client, index_name):\n",
    "    fields = [\n",
    "        SimpleField(name=\"id\", type=SearchFieldDataType.String, key=True),\n",
    "        SearchField(\n",
    "            name=\"text\",\n",
    "            type=SearchFieldDataType.String,\n",
    "            searchable=True,\n",
    "        ),\n",
    "        SearchField(\n",
    "            name=\"embedding\",\n",
    "            type=\"Collection(Edm.SByte)\",  # OData syntax for 8-bit signed integer\n",
    "            vector_search_dimensions=1024,\n",
    "            vector_search_profile_name=\"my-vector-config\",\n",
    "            # hidden=False, Use hidden=False if you want to return the embeddings in the search results\n",
    "        ),\n",
    "    ]\n",
    "\n",
    "    vector_search = VectorSearch(\n",
    "        profiles=[\n",
    "            VectorSearchProfile(\n",
    "                name=\"my-vector-config\",\n",
    "                algorithm_configuration_name=\"my-hnsw\",\n",
    "            )\n",
    "        ],\n",
    "        algorithms=[\n",
    "            HnswAlgorithmConfiguration(\n",
    "                name=\"my-hnsw\",\n",
    "                kind=VectorSearchAlgorithmKind.HNSW,\n",
    "            )\n",
    "        ],\n",
    "    )\n",
    "\n",
    "    index = SearchIndex(name=index_name, fields=fields, vector_search=vector_search)\n",
    "    client.create_or_update_index(index=index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Index Documents and Their Embeddings\n",
    "Finally, this function indexes the documents along with their int8 embeddings into Azure AI Search. For demonstration, document IDs are generated sequentially, but in practical applications, it's essential to use meaningful identifiers like database row ID, unique filenames, or any other unique metadata associated with the document.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def index_documents(search_client, documents, embeddings):\n",
    "    documents_to_index = [\n",
    "        {\"id\": str(idx), \"text\": doc, \"embedding\": emb}\n",
    "        for idx, (doc, emb) in enumerate(zip(documents, embeddings))\n",
    "    ]\n",
    "    search_client.upload_documents(documents=documents_to_index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the workflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "ename": "UnauthorizedError",
     "evalue": "status_code: 401, body: {'message': 'invalid api token'}",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mUnauthorizedError\u001b[0m                         Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[15], line 9\u001b[0m\n\u001b[0;32m      1\u001b[0m documents \u001b[38;5;241m=\u001b[39m [\n\u001b[0;32m      2\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mAlan Turing  was an English mathematician, computer scientist, logician, cryptanalyst, philosopher and theoretical biologist.\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m      3\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mAlbert Einstein was a German-born theoretical physicist who is widely held to be one of the greatest and most influential scientists of all time.\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m      4\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mIsaac Newton was an English polymath active as a mathematician, physicist, astronomer, alchemist, theologian, and author who was described in his time as a natural philosopher.\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m      5\u001b[0m     \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mMarie Curie was a Polish and naturalised-French physicist and chemist who conducted pioneering research on radioactivity\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m      6\u001b[0m ]\n\u001b[0;32m      8\u001b[0m \u001b[38;5;66;03m# Generate embeddings\u001b[39;00m\n\u001b[1;32m----> 9\u001b[0m embeddings \u001b[38;5;241m=\u001b[39m \u001b[43mgenerate_embeddings\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdocuments\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m     11\u001b[0m \u001b[38;5;66;03m# Initialize Azure Search Index Client\u001b[39;00m\n\u001b[0;32m     12\u001b[0m search_index_client \u001b[38;5;241m=\u001b[39m SearchIndexClient(\n\u001b[0;32m     13\u001b[0m     endpoint\u001b[38;5;241m=\u001b[39msearch_service_endpoint,\n\u001b[0;32m     14\u001b[0m     credential\u001b[38;5;241m=\u001b[39mcredential,\n\u001b[0;32m     15\u001b[0m     index_name\u001b[38;5;241m=\u001b[39mindex_name\n\u001b[0;32m     16\u001b[0m )\n",
      "Cell \u001b[1;32mIn[11], line 7\u001b[0m, in \u001b[0;36mgenerate_embeddings\u001b[1;34m(texts, input_type)\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(texts, \u001b[38;5;28mstr\u001b[39m):\n\u001b[0;32m      5\u001b[0m     texts \u001b[38;5;241m=\u001b[39m [texts]\n\u001b[1;32m----> 7\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[43mco\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43membed\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m      8\u001b[0m \u001b[43m    \u001b[49m\u001b[43mtexts\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtexts\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m      9\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m     10\u001b[0m \u001b[43m    \u001b[49m\u001b[43minput_type\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minput_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m     11\u001b[0m \u001b[43m    \u001b[49m\u001b[43membedding_types\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mint8\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m     12\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m     13\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m [embedding \u001b[38;5;28;01mfor\u001b[39;00m embedding \u001b[38;5;129;01min\u001b[39;00m response\u001b[38;5;241m.\u001b[39membeddings\u001b[38;5;241m.\u001b[39mint8]\n",
      "File \u001b[1;32mc:\\Users\\heidist\\azure-search-vector-samples\\.venv\\Lib\\site-packages\\cohere\\client.py:208\u001b[0m, in \u001b[0;36mClient.embed\u001b[1;34m(self, texts, images, model, input_type, embedding_types, truncate, request_options, batching)\u001b[0m\n\u001b[0;32m    205\u001b[0m textsarr: typing\u001b[38;5;241m.\u001b[39mSequence[\u001b[38;5;28mstr\u001b[39m]  \u001b[38;5;241m=\u001b[39m texts \u001b[38;5;28;01mif\u001b[39;00m texts \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m OMIT \u001b[38;5;129;01mand\u001b[39;00m texts \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m []\n\u001b[0;32m    206\u001b[0m texts_batches \u001b[38;5;241m=\u001b[39m [textsarr[i : i \u001b[38;5;241m+\u001b[39m embed_batch_size] \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(\u001b[38;5;241m0\u001b[39m, \u001b[38;5;28mlen\u001b[39m(textsarr), embed_batch_size)]\n\u001b[1;32m--> 208\u001b[0m responses \u001b[38;5;241m=\u001b[39m \u001b[43m[\u001b[49m\n\u001b[0;32m    209\u001b[0m \u001b[43m    \u001b[49m\u001b[43mresponse\u001b[49m\n\u001b[0;32m    210\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mresponse\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_executor\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmap\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m    211\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;28;43;01mlambda\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mtext_batch\u001b[49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mBaseCohere\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43membed\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m    212\u001b[0m \u001b[43m            \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[0;32m    213\u001b[0m \u001b[43m            \u001b[49m\u001b[43mtexts\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtext_batch\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    214\u001b[0m \u001b[43m            \u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    215\u001b[0m \u001b[43m            \u001b[49m\u001b[43minput_type\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minput_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    216\u001b[0m \u001b[43m            \u001b[49m\u001b[43membedding_types\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43membedding_types\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    217\u001b[0m \u001b[43m            \u001b[49m\u001b[43mtruncate\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtruncate\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    218\u001b[0m \u001b[43m            \u001b[49m\u001b[43mrequest_options\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrequest_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    219\u001b[0m \u001b[43m        \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    220\u001b[0m \u001b[43m        \u001b[49m\u001b[43mtexts_batches\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    221\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    222\u001b[0m \u001b[43m\u001b[49m\u001b[43m]\u001b[49m\n\u001b[0;32m    224\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m merge_embed_responses(responses)\n",
      "File \u001b[1;32mc:\\Users\\heidist\\azure-search-vector-samples\\.venv\\Lib\\site-packages\\cohere\\client.py:208\u001b[0m, in \u001b[0;36m<listcomp>\u001b[1;34m(.0)\u001b[0m\n\u001b[0;32m    205\u001b[0m textsarr: typing\u001b[38;5;241m.\u001b[39mSequence[\u001b[38;5;28mstr\u001b[39m]  \u001b[38;5;241m=\u001b[39m texts \u001b[38;5;28;01mif\u001b[39;00m texts \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m OMIT \u001b[38;5;129;01mand\u001b[39;00m texts \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m []\n\u001b[0;32m    206\u001b[0m texts_batches \u001b[38;5;241m=\u001b[39m [textsarr[i : i \u001b[38;5;241m+\u001b[39m embed_batch_size] \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(\u001b[38;5;241m0\u001b[39m, \u001b[38;5;28mlen\u001b[39m(textsarr), embed_batch_size)]\n\u001b[1;32m--> 208\u001b[0m responses \u001b[38;5;241m=\u001b[39m \u001b[43m[\u001b[49m\n\u001b[0;32m    209\u001b[0m \u001b[43m    \u001b[49m\u001b[43mresponse\u001b[49m\n\u001b[0;32m    210\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mresponse\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_executor\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmap\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m    211\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;28;43;01mlambda\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mtext_batch\u001b[49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mBaseCohere\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43membed\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m    212\u001b[0m \u001b[43m            \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[0;32m    213\u001b[0m \u001b[43m            \u001b[49m\u001b[43mtexts\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtext_batch\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    214\u001b[0m \u001b[43m            \u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    215\u001b[0m \u001b[43m            \u001b[49m\u001b[43minput_type\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minput_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    216\u001b[0m \u001b[43m            \u001b[49m\u001b[43membedding_types\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43membedding_types\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    217\u001b[0m \u001b[43m            \u001b[49m\u001b[43mtruncate\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtruncate\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    218\u001b[0m \u001b[43m            \u001b[49m\u001b[43mrequest_options\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrequest_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    219\u001b[0m \u001b[43m        \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    220\u001b[0m \u001b[43m        \u001b[49m\u001b[43mtexts_batches\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    221\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    222\u001b[0m \u001b[43m\u001b[49m\u001b[43m]\u001b[49m\n\u001b[0;32m    224\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m merge_embed_responses(responses)\n",
      "File \u001b[1;32mC:\\Python311\\Lib\\concurrent\\futures\\_base.py:619\u001b[0m, in \u001b[0;36mExecutor.map.<locals>.result_iterator\u001b[1;34m()\u001b[0m\n\u001b[0;32m    616\u001b[0m \u001b[38;5;28;01mwhile\u001b[39;00m fs:\n\u001b[0;32m    617\u001b[0m     \u001b[38;5;66;03m# Careful not to keep a reference to the popped future\u001b[39;00m\n\u001b[0;32m    618\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m timeout \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m--> 619\u001b[0m         \u001b[38;5;28;01myield\u001b[39;00m \u001b[43m_result_or_cancel\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfs\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpop\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    620\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m    621\u001b[0m         \u001b[38;5;28;01myield\u001b[39;00m _result_or_cancel(fs\u001b[38;5;241m.\u001b[39mpop(), end_time \u001b[38;5;241m-\u001b[39m time\u001b[38;5;241m.\u001b[39mmonotonic())\n",
      "File \u001b[1;32mC:\\Python311\\Lib\\concurrent\\futures\\_base.py:317\u001b[0m, in \u001b[0;36m_result_or_cancel\u001b[1;34m(***failed resolving arguments***)\u001b[0m\n\u001b[0;32m    315\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m    316\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m--> 317\u001b[0m         \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfut\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mresult\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtimeout\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    318\u001b[0m     \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m    319\u001b[0m         fut\u001b[38;5;241m.\u001b[39mcancel()\n",
      "File \u001b[1;32mC:\\Python311\\Lib\\concurrent\\futures\\_base.py:456\u001b[0m, in \u001b[0;36mFuture.result\u001b[1;34m(self, timeout)\u001b[0m\n\u001b[0;32m    454\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m CancelledError()\n\u001b[0;32m    455\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_state \u001b[38;5;241m==\u001b[39m FINISHED:\n\u001b[1;32m--> 456\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m__get_result\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    457\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m    458\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTimeoutError\u001b[39;00m()\n",
      "File \u001b[1;32mC:\\Python311\\Lib\\concurrent\\futures\\_base.py:401\u001b[0m, in \u001b[0;36mFuture.__get_result\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    399\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_exception:\n\u001b[0;32m    400\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m--> 401\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_exception\n\u001b[0;32m    402\u001b[0m     \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m    403\u001b[0m         \u001b[38;5;66;03m# Break a reference cycle with the exception in self._exception\u001b[39;00m\n\u001b[0;32m    404\u001b[0m         \u001b[38;5;28mself\u001b[39m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n",
      "File \u001b[1;32mC:\\Python311\\Lib\\concurrent\\futures\\thread.py:58\u001b[0m, in \u001b[0;36m_WorkItem.run\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m     55\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m\n\u001b[0;32m     57\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m---> 58\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m     59\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mBaseException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m exc:\n\u001b[0;32m     60\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mfuture\u001b[38;5;241m.\u001b[39mset_exception(exc)\n",
      "File \u001b[1;32mc:\\Users\\heidist\\azure-search-vector-samples\\.venv\\Lib\\site-packages\\cohere\\client.py:211\u001b[0m, in \u001b[0;36mClient.embed.<locals>.<lambda>\u001b[1;34m(text_batch)\u001b[0m\n\u001b[0;32m    205\u001b[0m textsarr: typing\u001b[38;5;241m.\u001b[39mSequence[\u001b[38;5;28mstr\u001b[39m]  \u001b[38;5;241m=\u001b[39m texts \u001b[38;5;28;01mif\u001b[39;00m texts \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m OMIT \u001b[38;5;129;01mand\u001b[39;00m texts \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m []\n\u001b[0;32m    206\u001b[0m texts_batches \u001b[38;5;241m=\u001b[39m [textsarr[i : i \u001b[38;5;241m+\u001b[39m embed_batch_size] \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(\u001b[38;5;241m0\u001b[39m, \u001b[38;5;28mlen\u001b[39m(textsarr), embed_batch_size)]\n\u001b[0;32m    208\u001b[0m responses \u001b[38;5;241m=\u001b[39m [\n\u001b[0;32m    209\u001b[0m     response\n\u001b[0;32m    210\u001b[0m     \u001b[38;5;28;01mfor\u001b[39;00m response \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_executor\u001b[38;5;241m.\u001b[39mmap(\n\u001b[1;32m--> 211\u001b[0m         \u001b[38;5;28;01mlambda\u001b[39;00m text_batch: \u001b[43mBaseCohere\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43membed\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m    212\u001b[0m \u001b[43m            \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[0;32m    213\u001b[0m \u001b[43m            \u001b[49m\u001b[43mtexts\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtext_batch\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    214\u001b[0m \u001b[43m            \u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    215\u001b[0m \u001b[43m            \u001b[49m\u001b[43minput_type\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minput_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    216\u001b[0m \u001b[43m            \u001b[49m\u001b[43membedding_types\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43membedding_types\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    217\u001b[0m \u001b[43m            \u001b[49m\u001b[43mtruncate\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtruncate\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    218\u001b[0m \u001b[43m            \u001b[49m\u001b[43mrequest_options\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrequest_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    219\u001b[0m \u001b[43m        \u001b[49m\u001b[43m)\u001b[49m,\n\u001b[0;32m    220\u001b[0m         texts_batches,\n\u001b[0;32m    221\u001b[0m     )\n\u001b[0;32m    222\u001b[0m ]\n\u001b[0;32m    224\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m merge_embed_responses(responses)\n",
      "File \u001b[1;32mc:\\Users\\heidist\\azure-search-vector-samples\\.venv\\Lib\\site-packages\\cohere\\base_client.py:1847\u001b[0m, in \u001b[0;36mBaseCohere.embed\u001b[1;34m(self, texts, images, model, input_type, embedding_types, truncate, request_options)\u001b[0m\n\u001b[0;32m   1837\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m BadRequestError(\n\u001b[0;32m   1838\u001b[0m         typing\u001b[38;5;241m.\u001b[39mcast(\n\u001b[0;32m   1839\u001b[0m             typing\u001b[38;5;241m.\u001b[39mOptional[typing\u001b[38;5;241m.\u001b[39mAny],\n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m   1844\u001b[0m         )\n\u001b[0;32m   1845\u001b[0m     )\n\u001b[0;32m   1846\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m _response\u001b[38;5;241m.\u001b[39mstatus_code \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m401\u001b[39m:\n\u001b[1;32m-> 1847\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m UnauthorizedError(\n\u001b[0;32m   1848\u001b[0m         typing\u001b[38;5;241m.\u001b[39mcast(\n\u001b[0;32m   1849\u001b[0m             typing\u001b[38;5;241m.\u001b[39mOptional[typing\u001b[38;5;241m.\u001b[39mAny],\n\u001b[0;32m   1850\u001b[0m             construct_type(\n\u001b[0;32m   1851\u001b[0m                 type_\u001b[38;5;241m=\u001b[39mtyping\u001b[38;5;241m.\u001b[39mOptional[typing\u001b[38;5;241m.\u001b[39mAny],  \u001b[38;5;66;03m# type: ignore\u001b[39;00m\n\u001b[0;32m   1852\u001b[0m                 object_\u001b[38;5;241m=\u001b[39m_response\u001b[38;5;241m.\u001b[39mjson(),\n\u001b[0;32m   1853\u001b[0m             ),\n\u001b[0;32m   1854\u001b[0m         )\n\u001b[0;32m   1855\u001b[0m     )\n\u001b[0;32m   1856\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m _response\u001b[38;5;241m.\u001b[39mstatus_code \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m403\u001b[39m:\n\u001b[0;32m   1857\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m ForbiddenError(\n\u001b[0;32m   1858\u001b[0m         typing\u001b[38;5;241m.\u001b[39mcast(\n\u001b[0;32m   1859\u001b[0m             typing\u001b[38;5;241m.\u001b[39mOptional[typing\u001b[38;5;241m.\u001b[39mAny],\n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m   1864\u001b[0m         )\n\u001b[0;32m   1865\u001b[0m     )\n",
      "\u001b[1;31mUnauthorizedError\u001b[0m: status_code: 401, body: {'message': 'invalid api token'}"
     ]
    }
   ],
   "source": [
    "documents = [\n",
    "    \"Alan Turing  was an English mathematician, computer scientist, logician, cryptanalyst, philosopher and theoretical biologist.\",\n",
    "    \"Albert Einstein was a German-born theoretical physicist who is widely held to be one of the greatest and most influential scientists of all time.\",\n",
    "    \"Isaac Newton was an English polymath active as a mathematician, physicist, astronomer, alchemist, theologian, and author who was described in his time as a natural philosopher.\",\n",
    "    \"Marie Curie was a Polish and naturalised-French physicist and chemist who conducted pioneering research on radioactivity\"\n",
    "]\n",
    "\n",
    "# Generate embeddings\n",
    "embeddings = generate_embeddings(documents)\n",
    "\n",
    "# Initialize Azure Search Index Client\n",
    "search_index_client = SearchIndexClient(\n",
    "    endpoint=search_service_endpoint,\n",
    "    credential=credential,\n",
    "    index_name=index_name\n",
    ")\n",
    "\n",
    "# Create or update the search index to include the embedding field\n",
    "create_or_update_index(search_index_client, index_name)\n",
    "\n",
    "# Initialize the SearchClient\n",
    "search_client = SearchClient(\n",
    "    endpoint=search_service_endpoint, \n",
    "    index_name=index_name, \n",
    "    credential=credential\n",
    ")\n",
    "\n",
    "# Index the documents and their embeddings\n",
    "index_documents(search_client, documents, embeddings)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Perform a Vector Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Title: Alan Turing  was an English mathematician, computer scientist, logician, cryptanalyst, philosopher and theoretical biologist.\n",
      "Score: 0.6248218\n",
      "\n",
      "Title: Albert Einstein was a German-born theoretical physicist who is widely held to be one of the greatest and most influential scientists of all time.\n",
      "Score: 0.5913683\n",
      "\n",
      "Title: Marie Curie was a Polish and naturalised-French physicist and chemist who conducted pioneering research on radioactivity\n",
      "Score: 0.57643753\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from azure.search.documents import SearchClient\n",
    "\n",
    "# Query for vector search\n",
    "query = \"foundational figures in computer science\"\n",
    "\n",
    "# Generate query embeddings\n",
    "# Use input_type=\"search_query\" for query embeddings\n",
    "query_embeddings = generate_embeddings(query, input_type=\"search_query\")\n",
    "\n",
    "search_client = SearchClient(search_service_endpoint, index_name, credential)\n",
    "\n",
    "vector_query = VectorizedQuery(\n",
    "    vector=query_embeddings[0], k_nearest_neighbors=3, fields=\"embedding\"\n",
    ")\n",
    "\n",
    "results = search_client.search(\n",
    "    search_text=None,  # No search text for pure vector search\n",
    "    vector_queries=[vector_query],\n",
    ")\n",
    "\n",
    "for result in results:\n",
    "    print(f\"Text: {result['text']}\")\n",
    "    print(f\"Score: {result['@search.score']}\\n\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
